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ABSTRACT 

Under  sponsorship  from  various  Department  of 
Defense  (DoD)  organizations  and  a  Cooperative  Research 
and  Development  Agreement  (CRADA)  with  the  U.S. 
Research,  Development  and  Engineering  Command, 
Army  Research  Laboratory  (RDECOM-ARL), 
HyPerComp  has  significantly  advanced  the  state  of  the  art 
of  time-domain,  broad  band  electromagnetic  simulations. 
The  TEMPUS  (Time-Domain  ElectroMagnetic  Parallel 
Unstructured  Simulator)  environment  is  a  complete  self 
contained  code  suite  that  includes  computer  aided  design 
(CAD)  geometry  creation  and  repair,  unstructured 
gridding  for  full-scale  targets  with  general  materials, 
scalable  parallel  code  architecture,  higher-order  accurate 
discontinuous  Galerkin  solvers  for  Maxwell's  equations, 
and  post  processing  utilities  for  solution  visualization  and 
extraction  of  final  results  like  bistatic  and  monostatic 
radar  cross  section  (RCS),  synthetic  aperture  radar  (SAR) 
images,  and  high-range-resolution  (HRR)  profiles.  The 
high-performance  TEMPUS  environment  is  well  suited 
for  modeling  a  variety  of  targets  and  electromagnetic 
problems  of  interest  to  the  U.S.  Army  such  as:  1)  high¬ 
speed  projectiles  with  subtle  surface  discontinuities, 
ridges  and/or  fins,  2)  ground-based  targets  such  as  tanks 
and  scud  missile  launchers,  and  3)  foliage  penetration  and 
ground  interaction  for  target  under  trees  (TUT).  Some  of 
the  physics-based  phenomenological  features  that  govern 
the  electro  magnetic  response  of  general  targets  are:  a) 
specular  reflection,  b)  creeping  waves,  c)  traveling  waves, 
d)  slow  moving  surface  waves,  e)  edge  diffraction,  f) 
singular  currents  at  surface  discontinuities,  g)  resonating 
gaps  and  cavities,  and  h)  general  material  response. 

1.  INTRODUCTION 

The  ability  to  accurately  predict  the  scattering  and 
radiation  behavior  for  broadband  (up  to  100  GHz) 
electromagnetic  illumination  over  complex  targets  with 
geometrical  details  including  surface  discontinuities, 
gaps,  cracks,  thin  edges,  cavities  and  embedded  antennae 
coupled  with  material  treatments  is  a  critical  technology 
need  for  DoD.  There  are  three  basic  approaches  to 


numerical  simulation  of  Maxwell’s  equations:  1)  high 
frequency  asymptotics,  which  treats  scattering  and 
diffraction  as  local  phenomena,  2)  solution  of  an  integral 
equation  (frequency  domain)  for  radiating  sources  on  (or 
inside)  the  scattering  body,  which  couples  all  parts  of  the 
body  through  a  multiple  scattering  process,  and  3)  the 
direct  integration  of  the  differential  form  of  Maxwell's 
equations  in  time.  Accurate  prediction  for  broadband 
electromagnetic  illumination  is  currently  too 
computationally  expensive  to  be  effectively  addressed  by 
multiple  applications  of  even  the  fastest  frequency 
domain  techniques.  The  integration  of  Maxwell’s 
equations  in  time  offers  the  most  direct  and  general 
solution  for  broadband  radar  scattering  and  propagation  of 
electromagnetic  pulses  in  real  materials.  Time-domain 
methods  may  be  the  best  approach  for  applications  such 
as  SAR  imagery  that  require  large  bandwidths  and 
hundreds  of  aspect  angles.  The  challenge  for  time-domain 
methods  has  been  to  maintain  global  accuracy  in  the 
phase  and  amplitude  of  waves  scattered  by  large,  complex 
structures.  This  requires  a  well-conditioned  time/space 
discretization  procedure  that  exhibits  high-order 
convergence,  fast  and  scalable  computation  with  memory 
efficiency,  and  numerical  fidelity  through  mathematical 
error  control. 

Many  of  the  earlier  time-domain  integration  methods 
suffered  serious  limitations  in  the  accuracy  with  which 
boundary  conditions  could  be  satisfied,  both  on  the  target 
and  at  the  outer  limits  of  the  computational  domain, 
leading  to  significant  numerical  discretization  errors 
(Warming  and  Beam,  1976;  Rowell  et  al.  1995).  Von 
Neumann  analysis  on  regular  grids  shows  the  limitations 
of  low-order  integration  schemes  in  propagating  waves 
over  distances  long  compared  to  a  wavelength.  The 
standard  second-order  upwind  approach  suffers  large 
dispersion  and  damping  effects  when  it  is  applied  directly 
to  cell  averages  of  the  fields  in  a  second-order  time 
integration  scheme,  such  as  the  one  devised  by  Warming 
and  Beam.  This  is  true  even  though  the  scheme  maintains 
second-order  accuracy  in  space  as  well.  The  impact  of 
these  errors  on  RCS  computational  accuracy  can  be 
significant,  especially  when  waves  scattered  strongly 
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from  different  parts  of  the  target  come  together  with 
comparable  amplitudes  elsewhere  on  the  target  surface,  or 
combine  in  the  far  field  with  large  relative  errors  in  phase. 

We  have  undertaken  the  development  of  a  state-of-the- 
art  time-domain  computational  electromagnetics  (CEM) 
technology  known  as  TEMPUS  (Time-Domain  EM 
Parallel  Unstructured  Simulator).  TEMPUS  includes  all 
aspects  of  a  CEM  simulation  including  CAD  geometry 
modeling  and  repair,  unstructured  gridding  for  full-scale 
targets  with  general  materials,  parallel  run  set  up  (for  PC- 
and  workstation  clusters)  and  higher-order  accurate 
solvers  for  Maxwell's  equations,  and  post  processing 
utilities  for  solution  visualization  and  extraction  of  final 
results  like  bistatic  and  monostatic  RCS,  SAR  images, 
and  HR R  profiles. 

The  numerical  method  at  the  heart  of  TEMPUS 
integrates  the  full  set  of  Maxwell’s  equations  in  general 
media,  optimized  for  the  evaluation  of  radar  signatures  for 
arbitrary  targets  that  may  be  up  to  a  few  hundred 
wavelengths  across  in  the  largest  dimension.  The  method 
is  an  implementation  of  the  discontinuous  Galerkin 
approach  developed  by  Cockburn  and  Shu  in  1998  for 
solving  hyperbolic  equations  in  the  time  domain  on 
unstructured  grids.  A  description  of  this  implementation 
may  be  found  in  the  recent  paper  by  Kabakian  et  al.. 
(2004)  Some  of  the  notable  qualities  of  this  higher-order 
approach  include  extremely  low  phase  errors,  the  ability 
to  preserve  accuracy  in  the  presence  of  highly  irregular 
unstructured  meshes,  straightforward  interfacing  with 
material  boundaries,  and  straightforward  parallelization. 

The  objective  of  this  paper  is  to  highlight  the 
capabilities  of  TEMPUS  in  simulating  various 
electromagnetic  features  and  to  illustrate  the  accuracy  of 
TEMPUS  results  using  comparisons  to  measured  data. 

1.1  Physics  of  Maxwell’s  equations 

The  integration  of  Maxwell’s  equations  in  time  offers 
the  most  direct  and  general  solution  for  broadband  radar 
scattering  and  propagation  of  electromagnetic  pulses  in 
real  materials.  The  challenge  for  time-domain  methods 
has  been  to  maintain  global  accuracy  in  the  phase  and 
amplitude  of  waves  scattered  by  large,  complex 
structures.  In  addition,  some  of  these  methods  have 
suffered  serious  limitations  in  the  accuracy  with  which 
boundary  conditions  could  be  satisfied  both  on  the  target 
and  at  the  outer  limits  of  the  computational  domain. 

The  sources  of  the  electromagnetic  fields  of  interest 
are  typically  currents  on  antenna  structures  and 
polarizations  induced  in  target  materials  by  incident  radar 
pulses.  The  simplest  case  of  plane- wave  reflection  by  a 
metal  aircraft  can  be  treated  as  a  problem  in  determining 
the  currents  induced  on  the  aircraft  surface.  Other  cases  of 


interest,  such  as  the  interaction  of  high-power  microwave 
pulses  with  shielding  of  various  kinds,  require  tracking 
the  buildup  of  field  amplitudes  inside  the  materials,  which 
can  trigger  electrical  breakdown  or  even  changes  in 
physical  state.  Such  processes  depend  nonlinearly  on  the 
local  field  strength,  and  consequently  they  are  not 
amenable  to  the  frequency-domain  description 
traditionally  employed  in  radar  scattering,  which  assumes 
that  the  target  response  is  linear  in  the  local  fields. 


1.2  High-Order  Discontinuous  Galerkin  Solver 

Scattering  from  many  targets  of  interest  to  the  DoD  is 
dominated  by  surface  traveling  waves  and  cavities,  which 
require  modeling  long  distance  wave  propagation  with 
high  accuracy.  This  very  challenging  requirement  for 
numerical  schemes  is  currently  met  by  employing  very 
fine  meshes,  which  can  quickly  exhaust  computational 
resources.  Numerous  numerical  experiments  have  shown, 
however,  that  high-order  methods  are  far  more  efficient 
for  solving  scattering  problems  that  require  this  high 
accuracy.  Indeed,  although  high-order  methods  are  more 
computationally  expensive  per  grid  cell  than  low-order 
methods,  they  require  fewer  cells  to  approximate  the 
solution  with  the  same  accuracy. 

Also,  singularities  in  the  exact  solution  can  cause 
errors  to  accumulate  globally  in  the  numerical  solution  for 
some  approaches.  Even  for  more  robust  schemes, 
resolving  the  surface  currents  in  the  vicinity  of  a  physical 
discontinuity  (edge,  corner,  and  tip)  may  be  required  to 
predict  the  strength  of  the  diffracted  fields  with  acceptable 
accuracy. 

Using  the  discontinuous  Galerkin  method,  HyPerComp 
has  successfully  developed  the  unstructured,  parallel, 
high-order  solver  TEMPUS,  representing  a  significant 
leap  in  the  state  of  the  art  in  time-domain  CEM.  The 
discontinuous  Galerkin  method  has  three  key  properties 
that  we  have  found  to  be  essential  for  performing  fast  and 
accurate  simulations  for  electrically  large  and  complex 
targets: 

1.  Unstructured  hybrid  grid  support  to  handle 
complex  realistic  target  geometries. 

2.  Highly  scalable  parallelization  to  exploit 
massively  parallel  computers. 

3.  High-order  discretization  to  model  accurately 
long  distance  propagation  and  complex  wave 
interactions. 

A  brief  description  of  the  discontinuous  Galerkin  method 
is  given  by  Kabakian  et  al.  (2004) 


2.  TEMPUS  VALIDATIONS 

To  assess  the  wave  propagation  properties  of  the 
discontinuous  Galerkin  method,  a  number  of  numerical 
experiments  have  been  performed  in  one-  and  two- 
dimensions.  Using  the  one-dimensional  wave  equation,  a 
Gaussian  pulse,  having  9  cells  across  its  width  at  half¬ 
maximum,  is  propagated  over  5000  cell- widths  with  the 
Galerkin  method  using  polynomials  of  degrees  zero,  one, 
and  two.  The  results  for  a  standard  second-order  accurate 
“RK4”  algorithm  (Rowell  et  al.,1995)  and  the  Galerkin 
schemes  (Kabakian  et  al.  2004)  are  shown  in  Figure  1. 


Galerkin-pl  Galerkin-p2 

Fig.  1 .  One  dimensional  wave  propagation. 


The  “RK4”  algorithm  has  distorted  the  shape  of  the 
pulse,  which  is  indicative  of  the  large  accumulation  of 
phase  errors.  In  contrast,  the  Galerkin-pO  solution 
exhibits  high  levels  of  dissipation.  Both  methods  require 
much  finer  meshes  to  preserve  accuracy  over  such  a  long 
propagation  distance.  But  the  Galekin-pl,  and  especially 
the  Galerkin-p2,  solutions  have  retained  the  pulse  shape 
remarkably  well.  Furthermore,  in  contrast  to  the  “RK4” 
scheme,  the  dominant  type  of  error  for  DG  is  dissipation, 
with  phase  error  always  remaining  very  low.  This  makes 
the  Galerkin  schemes  extremely  attractive  for  accurately 
simulating  the  constructive  and  deconstructive 
interferences  that  occur  in  problems  dominated  by 
traveling  waves  and  multiple-bounces. 

2.1  Creeping  Wave  Prediction 

While  the  case  of  a  sphere  is  geometrically  simple, 
the  TEMPUS  broad  band  capability  accurately  predicts 
the  creeping  wave  behavior  that  shows  up  as  a  late  time 
return  in  the  HRR  profile.  This  is  illustrated  for  a  sphere 
in  Figure  2.  The  broadband  RCS  results  and  the 
associated  HRR  profile  for  a  sphere  (radius  of  40"  or  1 
meter)  for  frequencies  from  0  to  2  GHz  are  shown.  The 


RCS  variation  as  a  function  of  frequency  is  compared 
with  the  Mie  series  solution.  The  HRR  profile  clearly 
shows  the  presence  of  a  creeping  wave.  After  the  first 
range  signal  that  corresponds  to  the  pulse  striking  the 
nose  of  the  sphere,  a  creeping  wave  signal  shows  a 
response  at  a  range  given  by  the  diameter+  *radius.  In 
the  HRR  profile  this  corresponds  to  a  response  located 
approximately  62.8"  from  the  center  of  the  sphere. 

Range  Profile  for  o  Sphere,  0-2  GHz  Pulse 
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Fig.  2.  Specular  and  creeping  waves  for  a  sphere. 

2.2  Traveling  Wave  RCS 

Usually,  any  long  and  thin  object  will  trigger  a 
traveling  wave  pattern  that  dominates  the  backscatter 
return  at  certain  grazing  incidence  angles.  Accurate 
prediction  of  traveling  wave  RCS  is  a  challenge  for 
numerical  methods.  The  high-order  formulation  inherent 
in  the  TEMPUS  solver  is  well  suited  for  computing  such 
traveling  wave  behavior.  Many  examples  are  shown  here 
to  illustrate  the  traveling  wave  RCS  prediction  by 
TEMPUS.  Figure  3  shows  RCS  results  for  a  business 
card  size  flat  plate  target  that  is  dominated  by  the 
presence  of  a  traveling  wave  at  grazing  angles.  Figures  4 
and  5  show  results  for  a  long  and  slender  10”  ogive  that 
exhibit  a  strong  traveling  wave  for  near  nose-on 
incidence.  The  backscatter  RCS  and  the  90°  bistatic  RCS 
predicted  by  TEMPUS  compare  quite  well  with  data 


measured  by  the  National  Radar  Test  Facility.  Figure  5 
shows  a  large  backscatter  RCS  due  to  a  traveling  wave 
that  occurs  at  an  incident  azimuth  angle  of  8°  which  is  not 
present  at  an  incident  angle  of  0°. 


(2.5-5  pts/L  HH-pol) 
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Fig.  3.  Monostatic  RCS  for  3.5  by  2  business-card  flat- 
plate  problem  using  TEMPUS-p2. 
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Fig.  4.  TEMPUS  accurately  predicts  the  traveling  wave 
for  a  10"  ogive. 
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Fig.  5.  Range  profile  showing  a  strong  traveling  wave 
return  for  an  incidence  angle  of  8°  from  nose-on. 


Figure  6  shows  the  range  profile  for  a  50-caliber 
bullet  at  near  nose-on  incidence.  The  measurements 
were  made  at  frequencies  between  32.4-35.6  GHz.  This 
results  in  a  down-range  resolution  of  1.85".  The 
measured  range  profile  and  the  TEMPUS  simulation  for 
the  same  bandwidth  compare  well.  A  simulation  was  also 
performed  using  20-GHz  of  bandwidth  and  a  range 
resolution  of  0.3".  The  range  profile  shows  two  peaks, 
one  corresponding  to  the  tip  and  one  corresponding  to  the 
back  end  of  the  bullet. 


Bullet,  Azimuth=0.5°  ,  Elevation=2.5°,  VV-pol 
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Fig.  6.  Range  profile  for  a  bullet  at  near  nose-on 
incidence  angle. 

23  Small  Geometrical  Features  on  Large  Targets 

One  of  the  challenges  of  CEM  is  to  be  able  to  model 
small  geometrical  features  on  a  large  target  (Walter 
1965).  These  include  gaps,  cracks,  shaip  edges,  and  thin 
wires.  While  it  may  be  possible  to  model  these  small 
regions  through  appropriate  basis  functions  that  represent 
the  singular  behavior  of  EM  fields  in  their  vicinity,  we  are 
exploiting  the  TEMPUS  high-order  capability  by  gridding 
the  large  and  small  geometrical  features  through 
appropriate  grid  resolutions  leading  to  very  small  grid 
cells  being  present  along  with  large  cells. 

The  class  of  missile-like  objects  is  of  interest  to  the 
Army.  Small  variations  in  surface  continuity  on  a  cone- 
cylinder  body  can  cause  dramatic  changes  in  backscatter 
RCS.  Three  simple  missile-like  geometries,  shown  in 
Figure  6,  were  tested  to  study  the  effect  of  such 
variations,  The  tip- a  geometry  has  a  90°  step  on  the 
cylinder  body  where  tip-b  has  a  step  with  a  rounded  edge.. 
Tip-c  has  a  step  on  the  front  cone  in  addition  to  the 
rounded  edge  step  of  tip-b.  Results  were  generated  for 
these  geometries  at  a  center  frequency  of  34  GHz  (Ka 
band).  Figure  7  shows  a  comparison  of  HRR  profiles  for 
tip-a  and  tip-b,  and  a  similar  one  shown  in  Figure  8  for 
tip-b  and  tip-c.  The  HRR  profile  clearly  shows  a  large 
drop  in  backscatter  return  from  the  rounded  edge  relative 


to  the  sharp  edge  .  Figure  9  shows  a  comparison  of  the 
backscatter  RCS  for  tip-a  with  measured  data  (Pizzillo 
and  Wellman,  2003).  The  comparison  is  quite  good 
despite  the  fact  that  the  CAD  file  geometry  was  not  an 
exact  representation  of  the  measured  target.  The  high 
frequency  code,  Xpatch,  did  not  predict  the  behavior  of 
such  subtle  surface  discontinuities  for  head-on  aspect 
angles  (C.  Kenyon  2004,  personal  communication). 


Fig.  6.  Three  cone -cylinder  geometries  that  were 
measured  and  simulated. 


Fig.  7.  Range  profile  for  a  14-GHz  bandwidth  (24-  40 
GHz)  clearly  shows  the  return  from  regions  of  surface 
discontinuity. 

2.4  Slow  Moving  Surface  Waves 

Missile-like  targets  exhibiting  a  certain  pattern  of 
surface  grooves  can  encounter  slow  moving  surface 
waves.  TEMPUS  can  demonstrate  the  presence  of  such 
slow  moving  surface  waves  for  a  simple  corrugated 
cylinder.  Depending  on  the  corrugation  geometry  (depth 
and  width  of  corrugation  and  number  of  corrugations  per 
wavelength),  the  degree  of  slowness  of  surface  waves  is 
altered.  Figure  10  shows  the  geometry  and  gridding  for  a 
corrugated  cylinder  along  with  a  simulated  range  profile 


for  a  32-36  GHz  pulse.  This  result  shows  a  return  peak 
well  past  the  end  of  the  corrugation  shown  in  yellow 
indicating  the  presence  of  a  slow  moving  surface  wave. 
Figure  11  shows  a  comparison  of  range  profiles  for  a 
cylinder  with  and  without  corrugations  of  a  certain  size. 
When  the  corrugation  is  removed,  the  range  profile  shows 
no  sign  of  a  slow  moving  wave.  This  phenomenology  has 
been  confirmed  with  measurement  results  provided  under 
the  CRADA.  Such  features  dominate  the  RCS  of  many 
realistic  slender  targets  but  are  often  overlooked  in  less 
accurate  simulations. 


Fig.  8  Range  profile  for  a  14-GHz  bandwidth  (24-  40 
GHz)  with  a  surface  discontinuity  on  the  cone  tip. 


frequency  {GHs) 


Fig.  9.  Comparison  of  RCS  vs  frequency  between 
measured  data  and  TEMPUS  for  Tip-a. 


(NIS),  ethernet  cables,  switches,  routers  and  hubs,  rack 
mounts  and  power  supply  for  fine-grain  clusters. 
HyPerComp  is  currently  installing  a  128-node  PC  cluster 
using  Pentium  IV,  2.2-GHz  motherboard  with  at  least  1 
Gb  memory/node.  We  are  also  documenting  the  lessons 
learned  in  configuring  and  setting  up  both  the  hardware 
and  software  for  PC  clusters. 

3.  CONCLUSION 


The  radar  cross  section  of  many  realistic  targets  is 
influenced  by  several  physics-based  electromagnetic 
phenomena  such  as  creeping  waves,  traveling  waves  and 
slow  moving  surface  waves  that  are  difficult  to  simulate 
and  require  a  very  accurate  prediction  capability  to 
reproduce.  Employing  higher-order  algorithms,  the  time- 
domain  environment  TEMPUS  has  demonstrated  these 
phenomena. 


Fig.  10.  TEMPUS  simulation  of  slow  moving  surface 
wave  on  a  corrugated  cylinder. 


Fig.  11.  Effect  of  corrugation  on  slow  moving  surface 
wave. 


2.5  Fine  Grain  Parallel  Environment 

The  TEMPUS  environment  is  designed  for  a  high 
level  of  efficiency  to  be  able  to  effectively  solve  large- 
scale  simulations  on  fine  grain  parallel  clusters.  More 
recently,  the  major  emphasis  is  on  developing  the  entire 
TEMPUS  simulation  process  from  start  to  finish  on  low 
cost  Linux-based  PC-clusters.  HyPerComp  is  working  on 
both  the  software  related  issues  such  as  compiler  options 
(Redhat  LINUX  versions  from  5.0  through  9.0  or  Solaris- 
86),  operating  system  versions,  porting  graphical  tools  to 
different  platforms,  appropriate  MPI  libraries  for  message 
passing,  version  control  and  software  security,  as  well  as 
hardware  issues  such  as  network  information  service 
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